Concerning the performance of the Orthogonal Space Time Block Code based Multiple-Input Multiple-Output (OSTBC-MIMO) system over independent but not necessarily identically distributed (i.n.i.d.) generalized-K fading channels, the system model of the OSTBC-MIMO system was established in i.n.i.d. generalized-K fading channel by adopting M-QAM modulation scheme. The equivalent Signal-to-Noise Ratio (SNR) of the receiver was approximated by the product of two variables that composed of multiple independent Gamma distributed random variables. On the basis of that, the probability density function expression of the equivalent SNR was derived as well as the expression of the average symbol error probability, channel capacity and outage probability. The simulation results show that, in addition to the parameter m, the parameter k also has a significant impact on the overall system performance, which produces the non-negligible differences on the system performance between the i.n.i.d. generalized-K fading channels and the i.i.d. generalized-K fading channels.

Common cooperative relay selection used outdated Channel State Information (CSI) to select relay, and accurate channel prediction can provide precise CSI for relay selection. The existing channel prediction methods often cannot adapt to the rapidly time-varying channel, or with high complexity. Therefore a channel prediction method based on slope prediction was proposed, which was applied to relay selection for cooperative communication system. This channel prediction method combined first-order linear polynomial curve fitting with Finite Impulse Response (FIR) Wiener prediction, by using FIR Wiener predictor to predict the slope of channel first-order linear function, cut the prediction range into several small enough time slices, and implemented continuous channel prediction within these small segments of time. At the same time, the Channel-Prediction based Relay Selection (CPRS) method was proposed, and performance analysis was carried out. Simulation results show that in the comparison experiments with the coordination system of Outdated-Channel based Relay Selection (OCRS), CPRS can make the system Bit Error Ratio (BER) decline 13% to 63%, with significant performance improvement effect.

Combined with fast convergence and high accuracy, a hybrid intelligent algorithm was proprosed to solve the parameter calibration of the nonlinear system. The results that computed by the improved least-square algorithm became the the original values of the genetic algorithm where gene range could be dynamically changed. The experiment shows that the hybrid algorithm is valid in practical application.